Tag: api

Playing with Grafana and weather APIs

~ Gonzalo Ayuso ~ 1 Comment

Today I want to play with Grafana. Let me show you my idea:

I’ve got a Beewi temperature sensor. I’ve been playing with it previously. Today I want to show the temperature within a Grafana dashboard.
I want to play also with openweathermap API.

Fist I want to retrieve the temperature from Beewi device. I’ve got a node script that connects via Bluetooth to the device using noble library.
I only need to pass the sensor mac address and I obtain a JSON with the current temperature

#!/usr/bin/env node
noble = require('noble');

var status = false;
var address = process.argv[2];

if (!address) {
    console.log('Usage "./reader.py <sensor mac address>"');
    process.exit();
}

function hexToInt(hex) {
    var num, maxVal;
    if (hex.length % 2 !== 0) {
        hex = "0" + hex;
    }
    num = parseInt(hex, 16);
    maxVal = Math.pow(2, hex.length / 2 * 8);
    if (num > maxVal / 2 - 1) {
        num = num - maxVal;
    }

    return num;
}

noble.on('stateChange', function(state) {
    status = (state === 'poweredOn');
});

noble.on('discover', function(peripheral) {
    if (peripheral.address == address) {
        var data = peripheral.advertisement.manufacturerData.toString('hex');
        out = {
            temperature: parseFloat(hexToInt(data.substr(10, 2)+data.substr(8, 2))/10).toFixed(1)
        };
        console.log(JSON.stringify(out))
        noble.stopScanning();
        process.exit();
    }
});

noble.on('scanStop', function() {
    noble.stopScanning();
});

setTimeout(function() {
    noble.stopScanning();
    noble.startScanning();
}, 2000);


setTimeout(function() {
    noble.stopScanning();
    process.exit()
}, 20000);

And finally another script (this time a Python script) to collect data from openweathermap API, collect data from node script and storing the information in a influxdb database.

from sense_hat import SenseHat
from influxdb import InfluxDBClient
import datetime
import logging
import requests
import json
from subprocess import check_output
import os
import sys
from dotenv import load_dotenv

logging.basicConfig(level=logging.INFO)

current_dir = os.path.dirname(os.path.abspath(__file__))
load_dotenv(dotenv_path="{}/.env".format(current_dir))

sensor_mac_address = os.getenv("BEEWI_SENSOR")
openweathermap_api_key = os.getenv("OPENWEATHERMAP_API_KEY")
influxdb_host = os.getenv("INFLUXDB_HOST")
influxdb_port = os.getenv("INFLUXDB_PORT")
influxdb_database = os.getenv("INFLUXDB_DATABASE")

reader = '{}/reader.js'.format(current_dir)


def get_rain_level_from_weather(weather):
    rain = False
    rain_level = 0
    if len(weather) > 0:
        for w in weather:
            if w['icon'] == '09d':
                rain = True
                rain_level = 1
            elif w['icon'] == '10d':
                rain = True
                rain_level = 2
            elif w['icon'] == '11d':
                rain = True
                rain_level = 3
            elif w['icon'] == '13d':
                rain = True
                rain_level = 4

    return rain, rain_level


def openweathermap():
    data = {}
    r = requests.get(
        "http://api.openweathermap.org/data/2.5/weather?id=3110044&appid={}&units=metric".format(
            openweathermap_api_key))

    if r.status_code == 200:
        current_data = r.json()
        data['weather'] = current_data['main']
        rain, rain_level = get_rain_level_from_weather(current_data['weather'])
        data['weather']['rain'] = rain
        data['weather']['rain_level'] = rain_level

    r2 = requests.get(
        "http://api.openweathermap.org/data/2.5/uvi?lat=43.32&lon=-1.93&appid={}".format(openweathermap_api_key))
    if r2.status_code == 200:
        data['uvi'] = r2.json()

    r3 = requests.get(
        "http://api.openweathermap.org/data/2.5/forecast?id=3110044&appid={}&units=metric".format(
            openweathermap_api_key))

    if r3.status_code == 200:
        forecast = r3.json()['list']
        data['forecast'] = []
        for f in forecast:
            rain, rain_level = get_rain_level_from_weather(f['weather'])
            data['forecast'].append({
                "dt": f['dt'],
                "fields": {
                    "temp": float(f['main']['temp']),
                    "humidity": float(f['main']['humidity']),
                    "rain": rain,
                    "rain_level": int(rain_level),
                    "pressure": float(float(f['main']['pressure']))
                }
            })

        return data


def persists(measurement, fields, location, time):
    logging.info("{} {} [{}] {}".format(time, measurement, location, fields))
    influx_client.write_points([{
        "measurement": measurement,
        "tags": {"location": location},
        "time": time,
        "fields": fields
    }])


def in_sensors():
    try:
        sense = SenseHat()
        pressure = sense.get_pressure()
        reader_output = check_output([reader, sensor_mac_address]).strip()
        sensor_info = json.loads(reader_output)
        temperature = sensor_info['temperature']

        persists(measurement='home_pressure', fields={"value": float(pressure)}, location="in", time=current_time)
        persists(measurement='home_temperature', fields={"value": float(temperature)}, location="in",
                 time=current_time)
    except Exception as err:
        logging.error(err)


def out_sensors():
    try:
        out_info = openweathermap()

        persists(measurement='home_pressure',
                 fields={"value": float(out_info['weather']['pressure'])},
                 location="out",
                 time=current_time)
        persists(measurement='home_humidity',
                 fields={"value": float(out_info['weather']['humidity'])},
                 location="out",
                 time=current_time)
        persists(measurement='home_temperature',
                 fields={"value": float(out_info['weather']['temp'])},
                 location="out",
                 time=current_time)
        persists(measurement='home_rain',
                 fields={"value": out_info['weather']['rain'], "level": out_info['weather']['rain_level']},
                 location="out",
                 time=current_time)
        persists(measurement='home_uvi',
                 fields={"value": float(out_info['uvi']['value'])},
                 location="out",
                 time=current_time)
        for f in out_info['forecast']:
            persists(measurement='home_weather_forecast',
                     fields=f['fields'],
                     location="out",
                     time=datetime.datetime.utcfromtimestamp(f['dt']).isoformat())

    except Exception as err:
        logging.error(err)


influx_client = InfluxDBClient(host=influxdb_host, port=influxdb_port, database=influxdb_database)
current_time = datetime.datetime.utcnow().isoformat()

in_sensors()
out_sensors()

I’m running this python script from a Raspberry Pi3 with a Sense Hat. Sense Hat has a atmospheric pressure sensor, so I will also retrieve the pressure from the Sense Hat.

From openweathermap I will obtain:

  • Current temperature/humidity and atmospheric pressure in the street
  • UV Index (the measure of the level of UV radiation)
  • Weather conditions (if it’s raining or not)
  • Weather forecast

I run this script with the Rasberry Pi crontab each 5 minutes. That means that I’ve got a fancy time series ready to be shown with grafana.

Here we can see the dashboard

Source code available in my github account.

Generating push notifications with Pushbullet and Silex

~ Gonzalo Ayuso ~ 1 Comment

Sometimes I need to send push notifications to mobile apps (Android or IOS). It’s not difficult. Maybe it’s a bit nightmare the first times, but when you understand the process, it’s straightforward. Last days I discover a cool service called PushBullet. It allows us to install one client in our Android/IOS or even desktop computer, and send push notifications between them.

Pushbullet also has a good API, and it allows us to automate our push notifications. I’ve play a little bit with the API and my Raspberry Pi – home server. It’s really simple to integrate the API with our Silex backend and send push notifications to our registered devices.

I’ve created one small service provider to enclose the API. The idea is to use one Silex application like this

use Silex\Application;
use PushSilex\Silex\Provider\PushbulletServiceProvider;

$app = new Application(['debug' => true]);

$myToken = include(__DIR__ . '/../conf/token.php');

$app->register(new PushbulletServiceProvider($myToken));

$app->get("/", function () {
    return "Usage: GET /note/{title}/{body}";
});

$app->get("/note/{title}/{body}", function (Application $app, $title, $body) {
    return $app->json($app['pushbullet.note']($title, $body));
});

$app->run();

As we can see we’re using one service providers called PushbulletServiceProvider. This service provides us ‘pushbullet.note’ and allows to send push notifications. We only need to configure our Service Provider with our Pushbulled’s token and that’s all.

<?php
namespace PushSilex\Silex\Provider;
use Silex\ServiceProviderInterface;
use Silex\Application;
class PushbulletServiceProvider implements ServiceProviderInterface
{
    private $accessToken;
    const URI = 'https://api.pushbullet.com/v2/pushes';
    const NOTE = 'note';
    public function __construct($accessToken)
    {
        $this->accessToken = $accessToken;
    }
    public function register(Application $app)
    {
        $app['pushbullet.note'] = $app->protect(function ($title, $body) {
            return $this->push(self::NOTE, $title, $body);
        });
    }
    private function push($type, $title, $body)
    {
        $data = [
            'type'  => $type,
            'title' => $title,
            'body'  => $body,
        ];
        $ch = curl_init();
        curl_setopt_array($ch, [
            CURLOPT_URL            => self::URI,
            CURLOPT_HTTPHEADER     => ['Content-Type' => 'application/json'],
            CURLOPT_CUSTOMREQUEST  => 'POST',
            CURLOPT_POSTFIELDS     => $data,
            CURLOPT_HTTPAUTH       => CURLAUTH_BASIC,
            CURLOPT_USERPWD        => $this->accessToken . ':',
            CURLOPT_RETURNTRANSFER => true
        ]);
        $out = curl_exec($ch);
        curl_close($ch);

        return json_decode($out);
    }
    public function boot(Application $app)
    {
    }
}

Normally I use Guzzle to handle HTTP clients, but in this example I’ve created a raw curl connection.

You can see the project in my github account

Building a simple API proxy server with PHP

~ Gonzalo Ayuso ~ 8 Comments

This days I’m playing with Backbone and using public API as a source. The Web Browser has one horrible feature: It don’t allow to fetch any external resource to our host due to the cross-origin restriction. For example if we have a server at localhost we cannot perform one AJAX request to another host different than localhost. Nowadays there is a header to allow it: Access-Control-Allow-Origin. The problem is that the remote server must set up this header. For example I was playing with github’s API and github doesn’t have this header. If the server is my server, is pretty straightforward to put this header but obviously I’m not the sysadmin of github, so I cannot do it. What the solution? One possible solution is, for example, create a proxy server at localhost with PHP. With PHP we can use any remote API with curl (I wrote about it here and here for example). It’s not difficult, but I asked myself: Can we create a dummy proxy server with PHP to handle any request to localhost and redirects to the real server, Instead of create one proxy for each request?. Let’s start. Problably there is one open source solution (tell me if you know it) but I’m on holidays and I want to code a little bit (I now, it looks insane but that’s me 🙂 ).

The idea is:

...
$proxy->register('github', 'https://api.github.com');
...

And when I type:

http://localhost/github/users/gonzalo123

and create a proxy to :

https://api.github.com/users/gonzalo123

The request method is also important. If we create a POST request to localhost we want a POST request to github too.

This time we’re not going to reinvent the wheel, so we will use symfony componets so we will use composer to start our project:

We create a conposer.json file with the dependencies:

{
    "require": {
        "symfony/class-loader":"dev-master",
        "symfony/http-foundation":"dev-master"
    }
}

Now 

php composer.phar install

And we can start coding. The script will look like this:

register('github', 'https://api.github.com');
$proxy->run();

foreach($proxy->getHeaders() as $header) {
    header($header);
}
echo $proxy->getContent();

As we can see we can register as many servers as we want. In this example we only register github. The application only has two classes:
RestProxy, who extracts the information from the request object and calls to the real server through CurlWrapper.

<?php
namespace RestProxy;

class RestProxy
{
    private $request;
    private $curl;
    private $map;

    private $content;
    private $headers;

    public function __construct(\Symfony\Component\HttpFoundation\Request $request, CurlWrapper $curl)
    {
        $this->request  = $request;
        $this->curl = $curl;
    }

    public function register($name, $url)
    {
        $this->map[$name] = $url;
    }

    public function run()
    {
        foreach ($this->map as $name => $mapUrl) {
            return $this->dispatch($name, $mapUrl);
        }
    }

    private function dispatch($name, $mapUrl)
    {
        $url = $this->request->getPathInfo();
        if (strpos($url, $name) == 1) {
            $url         = $mapUrl . str_replace("/{$name}", NULL, $url);
            $queryString = $this->request->getQueryString();

            switch ($this->request->getMethod()) {
                case 'GET':
                    $this->content = $this->curl->doGet($url, $queryString);
                    break;
                case 'POST':
                    $this->content = $this->curl->doPost($url, $queryString);
                    break;
                case 'DELETE':
                    $this->content = $this->curl->doDelete($url, $queryString);
                    break;
                case 'PUT':
                    $this->content = $this->curl->doPut($url, $queryString);
                    break;
            }
            $this->headers = $this->curl->getHeaders();
        }
    }

    public function getHeaders()
    {
        return $this->headers;
    }

    public function getContent()
    {
        return $this->content;
    }
}

The RestProxy receive two instances in the constructor via dependency injection (CurlWrapper and Request). This architecture helps a lot in the tests, because we can mock both instances. Very helpfully when building RestProxy.

The RestProxy is registerd within packaist so we can install it using composer installer:

First install componser

curl -s https://getcomposer.org/installer | php

and create a new project:

php composer.phar create-project gonzalo123/rest-proxy proxy

If we are using PHP5.4 (if not, what are you waiting for?) we can run the build-in server

cd proxy
php -S localhost:8888 -t www/

Now we only need to open a web browser and type:

http://localhost:8888/github/users/gonzalo123

The library is very minimal (it’s enough for my experiment) and it does’t allow authorization.

Of course full code is available in github.

Building a client for a REST API with PHP.

~ Gonzalo Ayuso ~ 4 Comments

Today we’re going to create a library to use a simple RESTfull API for a great project called Gzaas. Don’t you know what the hell is Gzaas? Gzaas is simple idea idea of a friend of mine (hi @ojoven!). A simple project that creates cool mesages on full screen. Then you can share the link in twitter, facebook and things like that. Yes. I now when I explain it, the first word that appear in our minds is WTF, but it’s cool, really cool.

Ok. The API is a simple RESTfull API, so we can use it with a simple curl interface. A few lines of PHP and it will run smoothly. But Gzaas is cool so we’re going to create a cool interface too. This days I’m involved into TDD world, so we’re going to create the API wrapper using TDD. Let’s start.

The first version of Gzaas API is a very simple one. It allow to create new messages, and we also can list the whole set of styles, patterns and fonts availables. The idea is build something as modular as we can. Problably (I hope so) Gzaas will include new features in a future, so we don’t want to build a monolitc library. We are going to start with fonts. As we can see in documentation we need to perform a GET request to get the full list of available fonts. There’s an important parameter (nowFeatured) to show only the featured fonts (the cool ones). 

use Gzaas\Api;
$fonts = Api\Fonts::factory()->getAll(Api::FEATURED);

So we are going to crate a simple test to test our font API. 

$this->assertTrue(count($fonts) > 0);

Whith this test we’re going to test fonts, but we also can test styles and patterns.

    public function testFonts()
    {
        $fonts = Api\Fonts::factory()->getAll(Api::FEATURED);
        $this->assertTrue(count($fonts) > 0);
    }

    public function testStyles()
    {
        $styles = Api\Styles::factory()->getAll(Api::FEATURED);
        $this->assertTrue(count($styles) > 0);
    }

    public function testPatterns()
    {
        $patterns = Api\Patterns::factory()->getAll(Api::FEATURED);
        $this->assertTrue(count($patterns) > 0);
    }

Ok now we are going to test the creation of a gzaas. We will use a random font, style and pattern:

$font    = array_rand((array) Api\Fonts::factory()->getAll(Api::FEATURED));
$style   = array_rand((array) Api\Styles::factory()->getAll(Api::FEATURED));
$pattern = array_rand((array) Api\Patterns::factory()->getAll(Api::FEATURED));

Let’s go:

$gzaas = new Api();
$url = $gzaas->setApiKey('mySuperSecretApiKey') // Get one valid at http://gzaas.com/project/api-embed/api-key
    ->setFont($font)
    ->setBackPattern($pattern)
    ->setStyle($style)
    ->setColor('444444')
    ->setBackcolor('fcfcee')
    ->setShadows('1px 0 2px #ccc')
    ->setVisibility(0)
    ->setLauncher("testin gzaas API");

As we can see our library has a fluid interface (I like fluid interface). It’s easy to implement. We only need to return the instance of the object (return $this;) in each setter function.

Gzaas API returns the url of the created gzaas when success, so $url variable must be different than empty string. Let’s test it:

$this->assertTrue($url != '');

To use this library we need to include the following files:

require_once __DIR__ . '/Api/ApiInterface.php'; 
require_once __DIR__ . '/Api/Network.php'; 
require_once __DIR__ . '/Api/ApiAbstract.php'; 
require_once __DIR__ . '/Api/Fonts.php'; 
require_once __DIR__ . '/Api/Patterns.php'; 
require_once __DIR__ . '/Api/Styles.php'; 
require_once __DIR__ . '/Api.php';

because of that there’s a simple bootstrap.php file to include all files. But we’re going to take a step forward. We’re going to create phar archive with the whole library inside. The creation of phar archive is very simple. We need a index.php file with the bootstrap (it must end with a __HALT_COMPILER();) 

include dirname(__FILE__) . '/Api/ApiInterface.php';
include dirname(__FILE__) . '/Api/Network.php';
include dirname(__FILE__) . '/Api/ApiAbstract.php';
include dirname(__FILE__) . '/Api.php';
include dirname(__FILE__) . '/Api/Fonts.php';
include dirname(__FILE__) . '/Api/Patterns.php';
include dirname(__FILE__) . '/Api/Styles.php';

__HALT_COMPILER();

and the program buildPhar will create our phar file:

#!/usr/bin/env php
buildFromDirectory($location);

And that’s all. Now we can use the bootstrap.php file to include the library or we can also include the phar file:

include 'Gzaas/Gzaas.phar';

You can fork the code at github here. Feel free to browse the code and tell us what do you think?