Monday August 28, 2006

Memcached In MySQL

The MySQL query cache has rarely been of much use to me since it's a pretty much just an optimization for read-heavy data. Furthermore, if you have a pool of query hosts (e.g. you're using MySQL replication to provide a pool of slaves to select from), each with its own query cache in a local silo, there's no "network effect" of benefitting from a shared cache. MySQL's heap tables are a neat trick for keeping tabular data in RAM but they don't work well for large data sets and suffer from the same siloization as the query cache. The standard solution for this case is to use memcached as an object cache. The elevator pitch for memcached: it's a thin distributed hash table in local RAM stores accessible by a very lightweight network protocol and bereft of the featuritus that might make it slow; response times for reads ands writes to memcached data stores typical clock in at single digits of milliseconds.

RDBMS-based caches are often a glorified hash table; a primary key'd column and value column. Using an RDBMS as a cache works but it's kinda overkill; you're not using the "R" in RDBMS. Anyway, transacting with a disk based storage engine that's concerned with ACID bookkeeping isn't an efficient cache. MySQL has the peculiar property of supporting pluggable storage backends. MyISAM, InnoDB and HEAP backends are the most commonly used ones. Today, Brian Aker (of Slashdot and MySQL AB fame) announced his first cut release of his memcache_engine backend.

Here's Brian's example usage:

mysql>  INSTALL PLUGIN memcache SONAME 'libmemcache_engine.so' ; create table foo1 (k varchar(128) NOT NULL, val blob, primary key(k)) ENGINE=memcache CONNECTION='localhost:6666';

mysql> insert into foo1 VALUES ("mine", "This is my dog");
Query OK, 1 row affected (0.01 sec)

mysql> select * from foo1 WHERE k="mine";
+------+----------------+
| k    | val            |
+------+----------------+
| mine | This is my dog |
+------+----------------+
1 row in set (0.01 sec)

mysql> delete  from foo1 WHERE k="mine";
Query OK, 1 row affected (0.00 sec)

mysql> select * from foo1 WHERE k="mine";
Empty set (0.01 sec)

Brian's release is labelled a pre-alpha, some limitations apply, your milage my vary, prices do not include taxes, customs or agriculture inspection fees.

What works

• SELECT, UPDATE, DELETE, INSERT
• INSERT into foo SELECT ...

What doesn't work

• Probably ORDER BY operations
• REPLACE (I think)
• IN ()
• NULL
• multiple memcache servers (this would be cake though to add)
• table namespace, right now it treats the entire server as one big namespace

The memcached storage plugin runs against the bleeding edge MySQL (Brian sez, "You will want to use the latest 5.1 tree"). What's most exciting about this is using it in combination with MySQL 5.x's support for triggers. A cache entry stored from a query result can be invalidated by a trigger on the row that provides the cache entry data. AFAIK, that's exactly how folks have been using pgmemcache in PostgreSQL but I haven't had a chance to mess with that yet. Anyway, check out Brian's list announcement and post about it, kudos to him for hacking on this, I imagine this will add a lot of value to the MySQL user community.

mysql   memcached   postgresql  

( Aug 28 2006, 07:18:13 AM PDT ) Permalink

Sunday August 27, 2006

Stupid Object Tricks

When I wrote about OSCON last month, I mentioned Perrin Harkins's session on Low Maintenance Perl, which was a nice review of the do's and don'ts of programming with Perl, I really didn't dig into the substance of his session. Citing Andy Hunt (from Practices of an Agile Developer):

When developing code you should always choose readability over convenience. Code will be read many, many more times than it is written. (see book site)

Perrin enumerated a lot of the basic rules of engagement for coding Perl that doesn't suck. Some of the do's and don'ts highlights:

Do's

• use strict
• use warnings
• use source control
• test early and often, specifically recommending Test::Class and smolder
• follow conventions when you can

...mostly no brainers and yet a lot of Perl programmers are oblivious to basic best practices.

Don'ts

• don't use formats (use sprintf!)
• don't mess with UNIVERSAL (it's the space-time continuum of Perl objects)
• don't define objects that aren't hashes ('cept inside outs)
• don't rebless an existing object into a different package (if you describe that as polymorphism in a job interview, expect to be shown the door real quick)

And so on.

The sad fact is that there are many ways to write bad Perl. I was amused to see Damian Conway and Larry Wall sitting in the second row as Perrin read off the indictments that so many Perl programmers are guilty of. On that last point, I can't even figure out why anyone would ever want to do that or why Perl supports it at all. This is ridiculous:

package Foo;

sub new {
  my $class = shift;
  my $data = shift || {};
  return bless $data, $class;
}

package main;

my $foo = Foo->new;
print ref $foo, "\n";
bless $foo, 'Bar';
print ref $foo, "\n"; 

For the non-Perl readers, create an instance of Foo ($foo), then change it to an instance of Bar, printing out the class names as you go. The output is:

Foo
Bar

Anyone caught doing this will certainly come back as a two headed cyclops in the next life.

I've been trying to increase my python craftiness lately. I first used python about 10 years ago (1996) at GameSpot, we used it for our homebrewed ad rotation system. I fiddled with python some more at Salon as part of the maintenance of our ultraseek search system. But basically, python has always looked weird to me and I've avoided doing anything substantial with it. Well, my interest in it is renewed because there is a substantial amount of legacy code that I'm presently eyeballing and, anyway, I'm very intrigued by JVM scripting languages such as Jython (and JRuby). I'm looking for a best-of-both-worlds environment, things-are-what-you-expect static typing and compile time checking on the one hand and rapid development on the other. I was really astonished to learn that chameleon class assignment like Perl's is supported by Python. Python is strongly typed in that you have to explicitly cast and coerce to change types (very unlike Perl's squishy contextual operators which does a lot of implicit magic). But Python is also dynamically typed, an object's type is a runtime assignment. This is gross:

class Foo:

  def print_type(self):
    print self.__class__

class Bar:

  def print_type(self):
    print self.__class__

if __name__ == "__main__":
  foo = Foo();
  foo.print_type();
  foo.__class__ = Bar
  foo.print_type();

In English, create an instance of Foo (foo), then change it to an instance of Bar, printing out the class names as you go. The output is:

__main__.Foo
__main__.Bar

(Python prefices the class name with the current namespace, __main__) Anyone caught doing this will certainly come back as a reptilian jackalope in the next life.

Of course, Java doesn't tolerate any of these shenanigans. Compile time complaints of "what, are you crazy?!" would surely come hither from javac. There's no setClass(Class):void method in java.lang.Object, thank goodness, even though there is getClass():Class. One of the key characteristics of a language's usefulness for agile development has to be its minimalization of astonishing results, quirky idioms and here-have-some-more-rope-to-hang-yourself behaviors. If you can't read your own code from last month without puzzling over it, how the hell are you going to refactor it quickly and easily next month? Will your collaborators have an easier time with it? Perl has rightly acquired the reputation of a "write once, puzzle forevermore" language. I haven't dug into whether Ruby permits runtime object type changing (that would be really disappointing). I'll dig into that next, clearly the rails developers emphasis on convention and configuration over code is aimed at reducing the surprises that coders can cook up. But that doesn't necessarily carry back to Ruby itself.

python   perl   ruby   java   code   agile programming   jython   jruby   oscon   oscon06  

( Aug 27 2006, 08:51:11 AM PDT ) Permalink