Using CouchDb as filesystem with PHP


One of the problems I need to solve in my clustered PHP applications is where to store files. When I say files I’m not speaking about source code. I’m speaking about additional data files, such as download-able pdfs, logs, etc. Those files must be on every node of the cluster. One possible approach to the solution is to use a distributed filesystem, rsync or maybe use a file-server mounted on every node. Another solution may be the usage of CouchDb. CouchDb has two great features to meet or requirements with this problem. It allows us to store files as attachments and it also allows to perform a great and very easy multi-master replica system.

The usage of CouchDB is pretty straightforward. It implements a RESTfull interface to perform every operations. So the only thing we need is a REST client. Zend Framework has a great one. We dont’t really need a library. We can easily perform REST requests with the PHP’s Curl’s extension. I’ve created two libraries for working with CouchDb one is a low-level HTTP client (with curl) and another is higher level one (it uses the HTTP Client) for CouchDB operations. You can read two post about those libraries here and here.

Now I want to extend the features of my library. I want to use CouchDB as file storage in PHP. Instead of using file functions (fopen, fwrite, fread, …) I want to use another ones and store/read files in CouchDB. For doing this I’ve refactored those two libraries into another one called Nov. I also have embraced namespaces so I will use them in the library. This means it’s only available with PHP 5.3.

Here you are a summary of the library. That’s not a complete UML graph. It’s only a diagram with the main features only with educational purpose.

summary

The best to show the library is with an example:

First I’m going to start with the basic usage of Nov\CouchDb library:

// Starting up the loader
require_once("Nov/Loader.php");
Nov\Loader::init();

use Nov\CouchDb;
$cdb = new CouchDb('localhost', 5984);
$cdb->db('users');
$nombre = $cdb->db('ris_users')->select('gonzalo')->asObject()->name;
$apellido = $cdb->db('ris_users')->select('gonzalo')->asObject()->surname;
echo "Hello {$nombre} {$apellido}.
";

To allow me the use of different CouchDb Databases and to put the Database configuration in one file. I use the following configuration class:

class NovConf
{
    const CDB1 = 'CDB1';
    const PG1  = 'PG1';

    public static $_dbs = array(
    	self::PG1  => array(
            'driver'   => 'pgsql',
            'dsn'      => 'pgsql:dbname=pg1;host=localhost',
            'username' => null
            'password' => null,
        ),
        self::CDB1  => array(
            'driver'   => 'couchdb',
            'host'     => 'localhost',
            'port'     => 5984,
            'protocol' => 'http',
            'username' => null,
            'password' => null,
        ),
    );
}

As you can see I use the same configuration file for my PDO drivers and CouchDb.

Now I can use:

require_once("Nov/Loader.php");
Nov\Loader::init();

use Nov\CouchDb;
$cdb = CouchDb::factory(NovConf::CDB1)->db('users');

try {
    $cdb->insert('xxx', array('name' => 'xxx'));
} catch (CouchDb\Exception\DupValOnIndex $e) {
    echo "Already created\n";
}

$data = $cdb->select('xxx')->asObject();
$cdb->update('xxx', array('name' => 'xxx1'));
$cdb->delete('xxx')->asObject();

And now finally the file storage part:

For storing the files I’ve taken one design decision. Every files will be stored into separate CouchDb document. That’s means one file, one document. There’s another possible approach. One CouchDb document can be one folder and store every files as attachments of this folder in the same document. But I prefer the idea of not to track folders. Only files. So each CouchDb document will have only one attachment.

That’s an example of one document in CouchDb

{
   "_id": "/home/gonzalo/aasa.txt",
   "_rev": "2-48b501a81c38fd84a3e0351917e64135",
   "path": "/home/gonzalo",
   "_attachments": {
       "aasa.txt": {
           "stub": true,
           "content_type": "application/octet-stream",
           "length": 12,
           "revpos": 2
       }
   }
}

There’s another usage script. Here we can see all the features together. We create files, update and delete them. Internally Nov\CouchDb\Fs uses a predefined CouchDb database called fs.

use Nov\CouchDb\Fs;
use Nov\CouchDb\Fs\Exception;
require_once ("Nov/Loader.php");
Nov\Loader::init();

echo "<pre>";
// create an instance from a factory method
$fs = Fs::factory(NovConf::CDB1);
// Now we're going to delete a file. If it doesn't exists will throw a FileNotFound exception
try {
    $fs->delete("/home/gonzalo/aaa.txt");
} catch (Exception\FileNotFound  $e) {
    echo $e->getMessage() . "\n";
}
// Now we are going to create a file.
// the second parameter 'true' means if the file doesn't exist will be created. Similar than 'r+'
try {
    $fs->open("/home/gonzalo/aaa.txt", true)
	->write("asasasasasas", "application/octet-stream");
} catch (Exception\FileNotFound $e) {
    echo $e->getMessage() . "\n";
} catch (Exception\WriteError $e) {
    echo $e->getMessage() . "\n";
} catch (Exception $e) {
    echo $e->getMessage() . "\n";
}
// We open the file
$res = $fs->open("/home/gonzalo/aaa.txt");

// we can get the length and the content type
echo $res->getLenght() . "\n";
echo $res->getContentType(). "\n";
// We move it to another location
$to = "/another/location";
$res->move($to);

$res = $fs->open($to);
// we flush the file to the browser
echo $res->raw();

// finally we delete it
$res->delete();
echo "</pre>";

I’ve also created an extra class to allow to dump files from filesystem to CouchDb and vice-versa.

require_once ("Nov/Loader.php");
Nov\Loader::init();
echo "<pre>";
// from filesystem to couchdb
\Nov\CouchDb\Fs\Utils::fs2cdb("/path/from/", NovConf::CDB1);
// from couchdb to filesystem
\Nov\CouchDb\Fs\Utils::cdb2fs(NovConf::CDB1, "/path/to/");
echo "</pre>";

And that’s all. You can download the source code with the examples here. The examples are under document_root/tests/couchdb/ folder. Remember you will need PHP5.3.

About these ads

About Gonzalo Ayuso

Web Architect specialized in Open Source technologies. PHP, Python, JQuery, Dojo, PostgreSQL, CouchDB and node.js but always learning.

Posted on August 30, 2010, in CouchDB, php, Technology. Bookmark the permalink. 7 Comments.

  1. Nice!

    Have you considered implementing this as a stream wrapper.

    I think using this as file_get_contents(“couchdb:///home/gonzalo/aaa.txt”) would be nice. :)

  2. Gustavo Salazar

    Hola Gonzalo!, muy bueno el articulo!
    Quería hacerte una pregunta si se puede, estamos teniendo un problema de conexión con couchdb desde PHP, nos da el siguiente error: ‘Could not connect to server at 127.0.0.1:5984: ’13: Permission denied”, este error lo obtenemos en Linux Centos 6, pero el mismo ejemplo en Windows funciona correctamente!, alguna idea, porque estamos despistados!

    • Gonzalo Ayuso

      El cliente (PHP) no se puede conectar al servidor (couchdb). Entiendo que el host donde se ejecuta php es el mismo que donde se ejecuta couchdb (ya que estas usando localhost). Yo me aseguraría en ver que puerto esta abierto haciendo un telnet desde el usuario donde se ejecuta php al servidor.

  1. Pingback: Gonzalo Ayuso’s Blog: Using CouchDb as filesystem with PHP | Development Blog With Code Updates : Developercast.com

  2. Pingback: Working with clouds. Multi-master file-system replication with CouchDB « Gonzalo Ayuso | Web Architect

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 952 other followers

%d bloggers like this: