Open Source software released by Sulake Dynamoid Oy

Igbinary

Igbinary is a drop in replacement for the standard PHP serializer. Instead of time and space consuming textual representation, igbinary stores PHP data structures in a compact binary form. Savings are significant when using memcached or similar memory based storages for serialized data.

But where does the name "igbinary" come from? There was once a similar project called fbinary but it has disappeared from the Internet a long time ago. Its architecture wasn't very clean either. IG is an abbreviation for a Finnish social networking site IRC-Galleria.

Features

  • Supports same data types as the standard PHP serializer: null, bool, int, float, string, array and objects.
  • __autoload & unserialize_callback_func
  • __sleep & __wakeup
  • Serializable -interface
  • Data portability between platforms (32/64bit, endianess)
  • Tested on Linux amd64, Mac OSX x86, HP-UX PA-RISC and NetBSD sparc64

Implementation

Storing complex PHP data structures like arrays of associative arrays in the standard serialized form is very space inefficient. Igbinary uses two strategies to minimize size of the serialized form.

  1. Strings are stored only once by using a hash table. Arrays of associate arrays with very verbose keys are stored compactly. Hashing adds some computational overhead but saves memory and bandwidth.
  2. Numerical values are stored in the smallest primitive data type available: 123 = int8_t, 1234 = int16_t, 123456 = int32_t ... and so on.

How to use?

First install the extension. Add the following lines to your php.ini:

# Load igbinary extension
extension=igbinary.so

# Use igbinary as session serializer
session.serialize_handler=igbinary

... and in your PHP code replace serialize and unserialize function calls with igbinary_serialize and igbinary_unserialize.

News

  • 2009-10-16 Version 1.0.2 released - Bugfixes
  • 2008-07-05 Version 1.0.1 released - unserialize_callback_func support - slight speedup when serializing scalars
  • 2008-06-25 First public version 1.0.0

License

Copyright (c) 2008 Sulake Dynamoid Oy All rights reserved.:

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

- Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

- Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

- Neither the name of the 'igbinary' nor the names of its contributors may
  be used to endorse or promote products derived from this software without
  specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.

GitHub

GitHub reprository can be found at http://github.com/phadej/igbinary/tree/master

PHP-Memcached-igbinary

Andrei Zmievski has written a PHP-memcached extension based on the libmemcached library, which is becoming the standard client library for communication with a memcached server. From version 1.0.4 php-memcached can serialize data using igbinary.

Intarray

PHP arrays are overkill for storing primitive integer values. They are slow and consume huge amounts of memory. Did I mention that serialization of PHP arrays adds a lot of overhead?

Intarray (Integer array) PHP extension is a space and time efficient tool for handling large int32_t arrays and performing lookups and set operations. Basically each intarray is just a PHP binary string and Intarray extension provides a bunch of functions for performing operations on it.

Some real world use cases with user ids:

Usage

Intarray has two modes of operation:

  1. Array - Values in random order. May contain duplicates.
  2. Set - Unique values in ascending order.

The latter mode enables use of binary search and set operations that utilize merge algorithm. It is up to the user to ensure that appropriate functions are used on each type of array.

Examples

<?php  // Binary search

       // Create an empty array full of zeroes
       $a = intarray_create(100);
       // Fill it with some values in ascending order
       for ($n = 0; $n < 100; $n++) intarray_set($a, $n, $n * 4);
       // Perform an O(log n) search and find index of value 188
       $index = intarray_binarysearch($a, 188);
       echo $index;
?>

... and we got 47

<?php // Union

      // Create two intarrays
      $a = intarray_create_from_array(array(1, 2, 3));
      $b = intarray_create_from_array(array(3, 4, 5));
      // Get a union of them
      $u = intarray_union($a, $b);
      // Dump to screen
      intarray_dump($u);
?>

... and we got { 1, 2, 3, 4, 5 }

Caveats

  • Data is not stored in network byte order (endianess). You must be cautious if different CPU architectures are involved.
  • Ordered data is not guarded against illegal modifications. Maybe next major version has an OO-style wrappers for the array.

License

Copyright (c) 2007-2009 Sulake Dynamoid Oy All rights reserved.:

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

- Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

- Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

- Neither the name of the 'intarray' nor the names of its contributors may
  be used to endorse or promote products derived from this software without
  specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.

Bugs & Contributions

To report bugs, or to contribute fixes and improvements send email to opensource@dynamoid.com .