Lenz Grimmer's blog

Even though MySQL is used to power a lot of web sites and applications that handle large binary objects (BLOBs) like images, videos or audio files, these objects are usually not stored in MySQL tables directly today. The reason for that is that the MySQL Client/Server protocol applies certain restrictions on the size of objects that can be returned and that the overall performance is not acceptable, as the current MySQL storage engines have not really been optimized to properly handle large numbers of BLOBs. To work around these limitations, these projects usually just store a reference to the object (e.g. a path name in a regular file system). This approach works around the limitations applied by the MySQL Server, but results in a disconnection and potential source of inconsistency between the database and the file system content. There was an interesting discussion about that topic on Sheeri's blog some time ago, with an excellent reply by Kristian, if you are curious to read more about this.

Thanks to the pluggable storage engine architecture of MySQL 5.1, we now see many independent storage engines emerging, one of them being PBXT by Paul McCullagh from SNAP Innovation. One of its strengths is going to be the handling of large binary objects in a performant way, so data objects can be stored in the database along with the other information related to it. To compliment this feature and to work around the limitations applied by the Client/Server Protocol, SNAP has now launched the Scalable BLOB Streaming Architecture for MySQL project, which intends to develop an architecture that accompanies the MySQL Client/Server protocol by providing an API to access and stream BLOBs stored in a MySQL database. See their press release (PDF) for the official announcement..

Paul will hold a BOF about this at the MySQL Conference and Expo this Tuesday at 19:30 in room San Thomas. So if storing and streaming large binary objects out of a MySQL Database is an essential requirement for your project, make sure to attend this session and provide Paul with your feedback and ideas!

Jay has written an excellent summary of sessions at the MySQL Conference & Expo that will be particularly interesting for developers using MySQL to power their applications or web sites. I noticed that he did not explicitely mentioned the Perl scripting language, even though there actually are a few sessions that will be of interest for the Perl Mongers amongst us (I would not dare to claim I am one myself, I just hack on it in my spare time):

• There will be a tutorial about "Real-world MySQL Performance Tuning", held by our very own Jay Pipes and Ask Bjørn Hansen. The tutorial will take place on Monday, 1:30pm in Ballroom B. As Ask is a very seasoned Perl hacker, I expect that he will spice up his part of the presentation with many valuable Perl hints as well. In any case, he's the one to talk to about all things Perl and MySQL!
• On Tuesday at 1:40pm, Earl Cahill from United Online will explain how they use Perl, Memcached and MySQL for "Logging Terabytes of Hits with MySQL". The session will take place in Ballroom A.
• Memcached and Perl seem to be a popular combination: Michael Otto and Erick Dennis from ePublica will explain their solutions to "Solving Growing Pains for OpenBC/Xing.com", which also involves a lot of Perl wizardry. Their talk will be on Tuesday, 2:30pm in Ballroom D. As Xing.com has grown their user base quite significantly and are now expanding their presence outside of Europe as well, I would have expected that their talk would also be referenced in this article about high profile success stories about scaling web sites, but somehow it was missing... In any case, this is going to be an interesting insight into Europe's largest social networking site.
• Giuseppe will also hold a BOF about "Using Perl's DBD::mysql". It will take place in the "Lawrence" room on Tuesday, at 7:30pm.
• Tobias Eggendorfer from the Universität der Bundeswehr in Neubiberg, Germany will speak about "Optimizing Performance for Dynamic, High Volume Web Pages" on Wednesday, 2:30pm in Ballroom D. They also utilize Perl to cope with the performance requirements.

I am sure I missed some relevant sessions, as Perl is usually taken as a given in many of today's implementations and applications and may therefore not be considered worthwhile to mention anymore. It's probably more hip to work with Python or Ruby these days... Let me know, if I missed one!

In closing I'd also like to add another shameless plug for the DotOrg Pavilion here: Perl is going to be present there, too: the very popular XAMPP application stack of course includes it, and Bugzilla is an entire application written in this scripting language. So make sure to stop by at the exhibition area and let them know, if you are a Perl geek as well!

In two weeks from now, the MySQL Conference & Expo 2007 will take place in Santa Clara, California. Our team has been very busy with working with the conference management team from O'Reilly to prepare for this event, especially Jay who is in charge of organizing and arranging the more than 110 sessions that will be held in several parallel tracks. The conference program looks very exciting, I am impressed by the wide range of topics and quality of speakers that we have lined up.

In addition to the presentations and tutorials, there will also be an Expo, where partner companies will exhibit their products related to MySQL. You will notice a number of well-known names in the industry there, e.g. hardware vendors like Dell, Dolphin Interconnect, HP or Sun as well as software companies like Alfresco, BitRock, Google, Pentaho, Red Hat and many more.

One of my tasks for this conference is to arrange the DotOrg Pavilion, an exhibition area inside the expo hall that is reserved for Open Source Community projects that are based on or support MySQL in some way. The conference web page does not yet list them all, therefore I'd like to give you a full rundown of all the OSS projects that will be present there:

• Bugzilla is an open source bug tracker used by thousands of businesses to track defects and enhancements. Bugzilla, the standard in open source bug tracking, is a versatile and powerful web based system with a minimal footprint, proven to support groups of any size from small projects to multinational enterprises.
• dotProject is a Web-based project management framework that includes modules for companies, projects, tasks (with Gantt charts), forums, files, a calendar, contacts, tickets/helpdesk, multi-language support, user/module permissions, and themes. It is translated into 17 languages and has a modular design that allows extra modules (such as time sheets and inventory) to be added in easily.
• Drupal is a powerful, scalable and full-featured content management system that allows an individual or a community of users to easily publish, manage and organize content and run an online community. Tens of thousands of people and organizations use Drupal to power scores of different web sites. A large, healthy and diverse community of developers, users and consultants drives a rapid pace of innovation and a wide variety of feature development.
• Eventum is user friendly and extensible issue tracking software from MySQL that can be used by a support department to track incoming technical support requests, a software development team to quickly organize tasks and bugs or anyone else looking to organize their work.
• HeidiSQL is an easy-to-use interface for developers using MySQL, allowing simple management of their databases and tables. Previously known as MySQL-Front, the name has been switched in 2006 by Ansgar Becker, the first developer, that put his old sources into a new open source project at SourceForge. Currently this tool is available on Windows and Linux.
• Joomla! is one of the most outstanding content management systems in the world. This is not only because of his approach of providing one of the most well structured and simple user interfaces for managing your media, text, image and other public information materials in the web. With Joomla! you are able to provide a huge information base for your professional audience as simple as you build up your own family community website. The new refactored content framework gives you the confidence that your site is based on a reliable, flexible, future proven and easy to manage plattform.
• LinuxQuestions.org is a free, friendly and active Linux Community with over 180,000 members and over 1,7000,000 posts. Founded in 2000 LQ offers forums, reviews, a Linux hardware compatibility list, a Linux knowledge base in wiki format, Linux tutorials, a free Linux download site, a Linux podcast, a Linux radio show and more. LQ has forums for everything from Linux Newbies to Linux in the Enterprise and has over 20 officially recognized Linux distribution forums.
• OTRS is an Open source Ticket Request System (also well known as trouble ticket system) with many features to manage customer telephone calls and e-mails. The system is built to allow your support, sales, pre-sales, billing, internal IT, helpdesk, etc. department to react quickly to inbound inquiries. Do you receive many e-mails and want to answer them with a team of agents? You're going to love the OTRS!
• Phorum, is the original PHP/MySQL message board software. Started in 1997 by Brian Moon, Phorum has been used by sites such as Zend.com and currently is the message board of choice for MySQL.com. Phorum's developers focus on speed and expandability through its flexible module system.
• phpMyAdmin is a tool written in PHP intended to handle the administration of MySQL over the Web. Currently it can create and drop databases, create/drop/alter tables, delete/edit/add fields, execute any SQL statement, manage keys on fields, manage privileges,export data into various formats and is available in 50 languages.
• SnapLogic is an Open Source project that enables developers to easily and rapidly integrate applications and other data sources through a Data Integration Network. SnapLogic embraces the simplicity and scalability of the Web to collapse the complexity of data integration for Internet-scale integration.
• Sphinx is fast and powerful full text search engine designed with indexing database content in mind. It offers high performance, good scalability (with multi-server distributed searching), great search quality, and other advanced features such as efficient filtering, sorting and grouping. Both native search APIs and a Pluggable MySQL Storage Engine search client are available, which makes Sphinx easy to integrate into existing applications.
• WordPress is a state-of-the-art semantic personal publishing platform with a focus on aesthetics, web standards, and usability. What a mouthful. WordPress is both free and priceless at the same time. More simply, WordPress is what you use when you want to work with your blogging software, not fight it.
• XAMPP is a free software package containing the Apache web server, MySQL database and necessary tools to use the PHP and Perl programming languages. Currently, XAMPP is available for Windows, Linux, Solaris and Mac OS X (the X in its name could stand for any one of these operating systems).

I hope I have gathered an interesting mix of projects and look forward to meeting with these people in person soon! If you attend the conference, please make sure to stop by the DotOrg pavilion and have a chat with the representatives from these projects!

Yesterday we had our sixth MySQL Meetup here in Hamburg - we had a nice crowd of 21 people. Erick Dennis and Michael Otto from epublica gave a presentation about their experiences with MySQL and scaling it to keep up with the growth of the Xing.com web site. The talk was a rehearsal for their presentation at the upcoming MySQL Conference and Expo in Santa Clara - it went very well and we had good discussions and comments during and after the session. I took some pictures during the meeting, which are now available on my picture gallery. The meeting was rounded up with excellent chinese food, thanks again for the folks at the Ni Hao for hosting us! I look forward to our next meeting, which will take place some time in May. If you have a suggestion for a presentation, please let me know!