OSS

Even though things have been a tad bit turbulent around here in the recent weeks, our engineers did not rest and churned out an impressive number of updates and new releases of the MySQL Server and related products.

Here's a quick summary of what we released this year so far (in chronological order):

• MySQL Server 5.1.42
• MySQL Server 5.5.1 Milestone 2 aka "Betony"
• MySQL Workbench 5.2.14 Beta 4
• MySQL Connector/J 5.1.11
• MySQL Proxy 0.8.0
• MySQL Workbench 5.2.15 Beta 5
• MySQL Server 5.0.90
• MySQL Server 5.1.43
• MySQL Cluster 6.3.31
• MySQL Cluster 7.0.11
• MySQL Cluster 7.1.1-beta

Last week, my colleagues Giuseppe, Kai and myself attended the SAPO Codebits event in Lisbon, Portugal. Codebits is an annual, invite-only hacking event, which went on for three days. The venue they chose this year was the "Cordoaria", a former rope factory located in the Belém district, close to the 25 de Abril Bridge (which is an impressive sight!). I have been told that the Cordoaria is the longest building in Portugal and I have no doubts about that! The building is so long that the crew used bicycles to get from one end to the other. I've taken a number of pictures from the event as well as from Lisbon itself, you can find them in this flickr set.

The organizers described this year's event as follows:

3 days. 24 hours a day. 600 attendees. Talks. Workshops. Lots of food and beverages. 24 hour programming/hacking competition. Quizz Show. Rock Band Contest. Lots of gaming consoles. More food. More beverages. More coding. Sleeping areas. More fun. An unforgettable experience.

I wholeheartedly agree, we had a great time! The conference started with sessions and presentations on a wide range of topics on the first two days. Afterwards, a 24-hour programming contest was held. I was invited to give two talks, one being my all-time favourite about "MySQL High Availability solutions" (slides, video), the other one was titled "Why you should be using a distributed version control system (DVCS) for your project" (video, slides). Both went quite well and the feedback I received was pretty positive. Giuseppe talked about "MySQL Schema Migration" (slides, video) and gave an "Introduction to Gearman" (video). Kai's talk was titled "Think before you develop" (video) and gave a nice roundup of tips and best practices for setting up and developing new web projects.

The Codebits session schedule was filled with amazing and interesting talks in four parallel tracks. Sometimes it was hard to choose – some other talks I attended and enjoyed:

• Robert Nyman: JavaScript - From Birth To Closure (slides, video)
• Glenn Jones: Re-using social media data (slides)
• Brian LeRoux: PhoneGap: Mobile App Developer Zero to Hero (video)
• Brian LeRoux: The State of JavaScript on Mobile Devices
• Walter Belgers: Becoming a lockpicker

Walter gave a lockpicking workshop after his presentation, which I attended as well. I was quite impressed (and a bit shocked) to find out how easy many locks can be opened this way! Later that evening there even was a live band named "Pornophonique" playing (one guy with a guitar, the other one using an Nintendo Game Boy for making music), but I missed that show as I was too busy opening more locks... Fortunately the concert and most of the sessions were recorded on video (in excellent quality) and are already available from the SAPO video pages. Kudos for this speedy service!

But this just matches my overall conclusion of this event: very well organized, great speakers and venue. Thanks to the organizers for having us, we really enjoyed our stay!

This blog post is a by-product of my preparation work for an upcoming talk titled "Why you should be using a distributed version control system (DVCS) for your project" at SAPO Codebits in Lisbon (December 3-5, 2009). Publishing these thoughts prior to the conference serves two purposes: getting some peer review on my findings and acting as a teaser for the actual talk. So please let me know — did I cover the relevant aspects or did I miss anything? What's your take on DVCS vs. the centralized approach? Why do you prefer one over the other? I'm looking forward to your comments!

Even though there are several distributed alternatives available for some years now (with Bazaar, git and Mercurial being the most prominent representatives here), many large and popular Open Source projects still use centralized systems like Subversion or even CVS to maintain their source code. While Subversion has eased some of the pains of CVS (e.g. better remote access, renaming/moving of files and directories, easy branching), the centralized approach by itself poses some disadvantages compared to distributed systems. So what are these? Let me give you a few examples of the limitations that a centralized system like Subversion has and how these affect the possible workflows and development practices.

So you're a small startup company, ready to go live with your product, which you intend to distribute under an Open Source License. Congratulations, you made a wise decision! Your developers have been hacking away frantically, getting the code in good shape for the initial launch. Now it's time to look into what else needs to be built and setup, so you're ready to welcome the first members of your new community and to ensure they are coming back!

Keep the following saying in mind, which especially holds true in the Open Source world: "You never get a second chance to make a first impression!". While the most important thing is of course to have a compelling and useful product, this blog post is an attempt to highlight some other aspects about community building and providing the adequate infrastructure. This insight is based on my own experiences and my observations from talking with many people involved in OSS startups and projects.

This time of the year is usually a very busy one, as there are plenty of events and conferences to attend. Just take a look at our calendar of OSS events on the MySQL Forge to see what I mean! Here's a quick summary of the ones that I will attend and speak at until the end of this year:

On November 14-15, I'll attend the openSQL Camp in Portland (OR), USA. I missed the first one that took place in Charlottesville (VA) in 2008, but had a lot of fun organizing the European Edition earlier this year. The upcoming one will be more like an unconference again - the list of proposed sessions looks very interesting and the attendee list reads like a "who is who" list of the OSS database community.

On December 3-5, I'll be joining Giuseppe at SAPO Codebits in Lisbon, Portugal, which is going to be a very cool event: "3 days. 24 hours a day. 600 attendees. Talks. Workshops. Lots of food and beverages. 24 hour programming/hacking competition. Quizz Show. Rock Band Contest. Lots of gaming consoles. More food. More beverages. More coding. Sleeping areas. More fun. An unforgettable experience". I will be talking about my favorite topic of MySQL High Availability (I'm currently working on revising my slides based on several excellent discussions about MySQL HA that happened on Planet MySQL in the past weeks) and about the benefits (both social and technical) of using a distributed revision control system (DVCS) like bazaar, git or mercurial for your open source project.

Shortly after Codebits, I will attend SLAC 09, the "Secure Linux Administration Conference" in Berlin, Germany (December 10-11), where I will give two MySQL-related talks (in German) - my usual suspects, but in revised and extended form: MySQL High Availability solutions and MySQL Backup & Security best practices.

FOSDEM, the Free and Opensource Conference, will again take place in Brussels, Belgium on Saturday and Sunday (6th and 7th February, 2010). Now happening for the 10th time (congratulations!), it is one of the largest Open Source conferences in Europe, with a strong focus on developers. Sun/MySQL have been regular sponsors of and contributors to the event in the past and it is alway a great experience to be there. It's very rare to meet so many well-known and bright people from such a wide range of OSS projects.

They have now opened their Call for Participation - the organizers are seeking input on talks for the main conference tracks (deadline: 2009-11-22) , lightning talks (deadline 2009-12-28) and project stands (deadline: 2009-11-22).

As for the last conference, we plan to apply for a MySQL developer room (and maybe a project stand, anybody interested to join? Please contact me!). However, the web site currently states that they are "slightly reworking the concept of developer rooms", so it remains to be seen what this will turn into. In any case, we will set up our own call for papers, once the developer room allocations have been finalized (and we were lucky). I was initially thinking about running another OpenSQL Camp, but it's probably too short after the upcoming one in Portland...

If you have an interesting talk about a MySQL-related subject, consider submitting your proposal now! We will gladly review and comment on your proposal in advance, if you would like to get our input or need suggestions about topics! Thanks.

With IntelliJ now being available under an Open Source license, developers have another option to choose from when it comes to Java-based IDEs/Frameworks (Eclipse and NetBeans being the other two prominent ones). Choice is always good, and being an Open Source enthusiast, I of course welcome JetBrain's move!

However, as I'm not really a heavy GUI-based IDE user myself, I can't really comment on which one is the best. These kind of discussions tend to turn into a Holy War anyway... In the end it's likely that each of them gets the job done and you have to come to your own conclusions, based on your personal preference and requirements.

I personally would be interested in seeing how their support for PHP or Python compares to the one in NetBeans. Their plugin repository lists more that 560 plugins, including many for database connectivity/modeling/navigation (incl. support for MySQL). I'm also glad to see that they have a plugin for Bazaar, something that I'm desperately missing from NetBeans!

Interestingly, they decided to keep a few parts proprietary, it's going to be interesting to see how this will turn out for them and if developers will be willing to pay for these extra features, considering that most of this is available for free from the other two projects.

Their Contributor License Agreement looks like it has been derived from the Sun Contributor Agreement (SCA), which is always nice to see. I assume this can be attributed to Roman Strobl - I was positively surprised to notice that he joined their team as a technology evangelist in June! Roman did a great job in spreading the NetBeans and OpenSolaris gospel at Sun before and I briefly met him at this year's FOSDEM conference in Brussels. Congratulations!

The summer break seems to be over and the event season is heating up again! There is a number of conferences and events coming up in the next months — here is a quick summary of the events that I plan to attend.

This Friday I will attend an event here in Hamburg: the "Silpion Sommerfest", organized by Silpion (a local IT solutions provider which is a partner of Sun Microsystems as well). I will be there to network and talk about MySQL.

This coming weekend (2009-09-12/2009-09-13), there will be the PHP Unconference here in Hamburg, Germany . It will consist of two days of Barcamp-style sessions about PHP. Sun/MySQL are sponsors of the event and I expect several of my team mates to be there as well. With more than 180 participants, the event is already sold out.

The following week I will be attending the openSUSE Conference in Nuremberg, Germany on 2009-09-17/2009-09-20. I will give the opening keynote on Thursday morning. Titled "Working in a Virtual Community", I will talk about the pros and cons of working in a virtual organization, giving an overview about some of the technical and social aspects that play a role in working with virtual communities.

On December 10th and 11th I will be attending the 4. Secure Linux Administration Conference 2009 (SLAC) in Berlin, Germany. I've been invited to talk about MySQL and will give two sessions about MySQL Backup & Security as well as MySQL High Availability Solutions. The Call for Papers for this event is still open, so if you have a technical, "best practice" talk that might be relevant for system administrators, consider submitting your proposal!

I am happy to announce that mylvmbackup version 0.13 has now been released. This release includes a fix for a nasty bug in on of the recently added Perl hooks (precleanup.pm) and some added functionality (better support for remote rsync backups).

From the ChangeLog: 

• Deleted sample precleanup.pm hook as it has potential to cause harm and is too specialized on a particular use case (BUG#394668)
• Added support for rsync via SSH (BUG#392462)
• Fixed InnoDB recovery in case a relative path to the MySQL data directory is defined (BUG#38337), improved the documentation of relpath in the man page.

It's almost two weeks now since FrOSCon and the OpenSQL Camp subconference have taken place in Sankt Augustin, Germany — about time for a summary and update from my side!

First off, I would like to thank all of the participants and supporters, particularly my colleagues Regina Steyer and Iris Musiol for the perfect logistics and co-sponsoring as well as Uli Graef, Thorsten Frueauf, Matthias Schmidt, Alexander Rubin and Joerg Moellenkamp for manning the Sun booth and the help on site.

Another big Thank You goes out to my team mates Giuseppe and Colin as well as to Sheeri K. Cabral, who were a big help in keeping the OpenSQL Camp on track and by supporting the event by giving talks. In addition to that, Sheeri recorded most of the OpenSQL Camp sessions on video and published them in record time!

So here's a quick summary of both events from my side, starting with the main conference.

Sun was a Gold sponsor of the event and we had a booth right at the main entrance area; it could hardly be missed. It consisted of two large and two small desks as well as a divider behind them. For demos, we had a (slightly noisy) Sun Fire X2200 M2 Server and four SunRay 2 Thin Clients (which by themselves triggered a lot of questions and curiosity by many visitors). The booth was flanked by rollup-banners on both sides as well as various posters attached to the divider. Here's a picture of our booth before the event opened:



We demoed Open Solaris, Open HA Cluster, NetBeans/Java and MySQL. We also had a lot of brochures about various products, OpenSolaris 2009.06 Live-CDs incl. booklets as well as some MySQL-T-Shirts to hand out. We distributed over 300 CDs and received a lot of positive feedback about the distribution.

We also had a number of talks in the main conference track (both German and English):

• Lenz Grimmer: Working for a Virtual Company - How we do it at MySQL (OGG Video)
• Giuseppe Maxia: MySQL Sandbox 3 - Making your life with databases easier than ever (OGG Video)
• Ulrich Graef: ZFS - Neue Funktionen und Technologien
• Thorsten Früauf: Hochverfügbarkeit mit minimalem Cluster - Open HA Cluster on OpenSolaris (OGG Video)
• Jörg Möllenkamp: Insights to Opensolaris (OGG Video)

The comments and ratings of these sessions were generally very positive. Our booth was well attended, especially during the session breaks. In total, there were over 1.400 visitors at the conference over the two days.

I personally did not attend many sessions in the main conference tracks, as I was too occupied with the OpenSQL Camp and the booth organization. However, I managed to listen to Uli Graef's talk, which was a very technical and interesting session about ZFS features and internals. Being a big fan of ZFS myself, this was a very worthwhile session to be at and my impression was that it encouraged others to take a closer look at this truly amazing file system.

The second talk I attended was Sunday's keynote by Dries Buytaert from the Drupal project about "The Secrets of Building and Participating in Open Source
Communities". Dries is a great speaker with visually stunning slides. He is funny, too — if you have a moment, you should watch the video recording of his keynote. An uncut "pre-release" version of his talk is already available as an OGG Video file.

As for previous FrOSCons (is that the proper plural?), there was a social event scheduled for Saturday evening, providing barbecue (Steaks and Sausages as well as vegetarian dishes) and drinks. This event usually takes place outside and is always an excellent opportunity for networking and talking with key people from other OSS communities and projects. And there was plenty of time for talking - the queues for the grilled food were long...

Here is a list of other blogs and articles about FrOSCon that are worth a read (in no particular order and both German and English):

• http://www.c0t0d0s0.org/archives/5878-FrOSCon-2009-first-day.html
• http://www.c0t0d0s0.org/archives/5879-FrOSCon-09-Nachlese.html
• http://blogs.sun.com/TF/entry/hochverf%C3%BCgbarkeit_mit_minimalem_cluster
• http://robilad.livejournal.com/53194.html
• http://blogs.sun.com/bblfish/entry/froscon_the_free_and_open
• http://www.linux-magazin.de/content/view/full/43032
• http://community.oreilly.de/blog/2009/08/25/das-war-die-froscon-2009/
• http://www.naturalnik.de/wordpress/2009/08/froscon-2009/
• http://www.alexander-stelter.de/blog/archives/361-FrOSCon-2009-Review-+-Bilder.html

OpenSQL Camp, European Edition

In addition to the main conference tracks, FrOSCon also provided a number of so-called "Developer Rooms" to OSS projects, so that they could organize sub-conferences or hackfests of their own. We applied for a room to set up a conference dubbed "OpenSQL Camp", related to the topic of Open Source databases, which was approved.

We then sent out a call for papers and invited people from the many OSS database communities to join us and talk about their projects. Every session proposal was published on the OpenSQL Camp web site and people were able to vote on the sessions they were most interested in via email or twitter:

• http://opensqlcamp.org/Events/2009/Proposed_Sessions
• http://opensqlcamp.org/Events/2009/Reviewing_and_Voting

The organization and scheduling of the talks and speakers was done via the FrOSCon conference system (Pentabarf), which made it very easy to perform this task and also made sure that the OpenSQL Camp sessions were included in the main conference program. Below is a full list of sessions at our subconference (see the FrOSCon Program page for abstracts, speaker info, links and slides). We had two cancellations by speakers on short notice, but were able to cover the gaps with ad-hoc presentations. I'd like to send a special thanks to Geert Vanderkelen, who gave a great presentation about MySQL Cluster despite the very short notice and some technical difficulties at the beginning!

• Sheeri K. Cabral: A Better mysqltuner (Video)
• Linas Virbalas: Bringing Master/Slave into the 21st Century using Tungsten Database Clustering (Video)
• Holger Klemt: Firebird - a really free database used in free and commercial projects (Video)
• Seppo Jaakola: Galera Replication (Video)
• Kristian Waagan: Getting acquainted with Apache Derby (Video)
• Stephane Combaudon: Minimizing data access with covering indexes
• Lenz Grimmer: MySQL High Availability Solutions (Video)
• Jan Kneschke: MySQL Proxy: a MySQL toolbox (Video)
• Felix Schupp: New kid on the block: The BlackRay Data Engine (Video)
• Vladimir Kolesnikov: PBXT: Technology trends that affect your database (Video)
• Panel Discussion: The OSS Toolshed Shootout (Video)
• Darren Cassar: Securich - MySQL user administration and security made easy!
• Giuseppe Maxia: Sharding for the masses (Video)
• Geert Vanderkelen: An introduction to MySQL Cluster

Most talks attracted between 20-50 attendees and we had a great mix of topics from several different database projects (with a slight majority of MySQL-related talks). The Panel Discussion (moderated by me), called the "OSS Toolshed Shootout" went quite well and the speakers had a good time answering questions on various topics about their projects. Thanks again to all OpenSQL Camp speakers for making this event a success!

All in all I think that both FrOSCon and OpenSQL Camp were well worth supporting and attending - we were able to provide insight and trigger some interesting discussions among the OSS enthusiasts and developers in the audience. It was also a good opportunity in get in touch with many people of other OSS communities, fostering the MySQL (and other Sun OSS projects) ecosystem.

Here is a Flickr slide show of my own pictures - more photos can be found in the FrOSCon Gallery and the links page on the Wiki.

I personally look forward to next year's FrOSCon - a Big Thanks to the organizers for another great event!

IPC 2009, the International PHP Conference will take place on November 15th-18th in Karlsruhe, Germany. While the deadline for the call for papers for the main conference program has already passed, there is still an opportunity to submit MySQL-related content: the organizers plan to have a special MySQL Day, which will take place on Tuesday, 17th of November.

Quoting for the Call for Papers web page:

We are looking for speakers joining the Architecture Day or the MySQL Day. "Architecture", in terms of project organization, business organization, tools & approaches etc. is becoming a key qualification to developers and teams. And MySQL still is one of the most common open source databases used in many of todays leading web applications. The International PHP Conference will provide attendees with professional and up to date information on both MySQL and Architecture in the world of PHP.

We are looking forward to you numerous suggestions and proposals, as usual via http://phpconference.com/input.

We would like to encourage you to support and contribute to this event by submitting interesting talks related to MySQL and PHP! If you have any questions or comments about this MySQL Day, feel free to get in touch with Robert Lippert or Björn Schotte directly. Thank you!

There have been several similar articles to this one, but I find it pretty amazing to see all these emerging and established Solaris/OpenSolaris distributions. It's a good sign of a healthy community!

So here is my collection - please let me know if I forgot one.

Maintained by Sun Microsystems:

• Solaris 10: The commercial, production-ready and full supported version, similar to Red Hat Enterprise Linux (RHEL) or SUSE Linux Enterprise Server (SLES).
• OpenSolaris: This is the "official" OpenSolaris distribution, developed & supported by Sun. Can be compared to Fedora or openSUSE Linux - this is the version which will become the basis of future "Enterprise" releases with a different support and maintenance model. The developers aim on publishing a release roughly every six months.
• Solaris Express: Community Edition (SX:CE): is Sun's periodically built unsupported distribution of OpenSolaris (along with many additional open-source and a few closed-source components necessary to make a complete installable system). However, it looks as if Sun plans to discontinue the production of this version soon.
• Solaris Express: Sun's official distribution of OpenSolaris. These are provided primarily for customers interested in trying out new features of the Solaris release currently under development without the total cutting-edge-ness of the Solaris Express: Community Edition

Third-party distributions (in alphabetical order):

• AuroraUX: A core operating system for high integrity scientific computing. AuroraUX is a Solaris-derived kernel- and user- land. The core of the project are its utilities written in Ada. When necessary, poorly implemented features get fixed or rewritten, as well.
• Belenix: BeleniX is an OpenSolaris Distribution with a Live CD (runs directly off the CD). It can be installed to harddisk and is free to use, modify and distribute.
• Embedded Operating system/Networking (EON): A RAM-based live ZFS NAS appliance (CIFS/NFS/Samba) that boots from USB.
• Jaris: An OpenSolaris derivative suitable for Japanese users, providing a windows-like environment. (The web site is available in Japanese only)
• Korona: A live DVD distribution based on OpenSolaris, but using KDE 4.3 as the default desktop.
• Martux: A barebones OpenSolaris distribution for SPARC
• MilaX: MilaX is a small size Live CD distribution which runs completely off a miniCD, bootable business card or USB flash drive.
• Nexenta Core: Basically an OpenSolaris Kernel with an Ubuntu/GNU userland (including an improved version of apt-get that utilizes ZFS snapshots to perform rollbacks of failed upgrades)
• SchilliX: The very first independent OpenSolaris-based distribution (seems like it's not maintained anymore)
• StormOS: an OpenSolaris desktop distribution, based on Nexenta Core

Some time ago I posted a compilation of applications and programming languages that provide an API to connect to the MySQL Server. As it turned out, I forgot a few that I would like to mention here:

• Apache DBD API: a MySQL driver for mod_apr_dbd is not included in the official distribution, but can be obtained seperately from here. Some distributions (e.g. openSUSE) actually provide installable packages of this driver module.
• GRASS MySQL driver: The MySQL database driver in GRASS enables GRASS applications to store vector attributes in a MySQL server. Alternatively, data can be stored in an embedded version of the MySQL Server.
• HDBC: HDBC provides an abstraction layer between Haskell programs and SQL relational databases. This lets you write database code once, in Haskell, and have it work with any number of backend SQL databases.
• libdrizzle: This is the client and protocol library for Drizzle and MySQL that provides both client and server interfaces. A number of APIs (e.g. Java, Lua, Perl and Ruby as well as Python) have been built on top of it, but I'm not quite sure if they all support the MySQL client/server protocol as well.
• MySQL Connector/Python: MySQL Connector/Python is implementing the MySQL Client/Server protocol completely in Python. This means you don't have to compile anything - the MySQL client library doesn't even have to be installed on your system.

Do you know of any other MySQL Connectors/APIs? Let me know!

I'm happy to announce that the schedule for OpenSQL Camp 2009 (European Edition) has been published on the FrOSCon timetable now. We have a great selection of topics and speakers, so don't miss it! OpenSQL Camp is a subconference of FrOSCon, the Free and Open Source Conference, which will take place on August 22nd and 23rd in St. Augustin, Germany.

The admission fee for the entire conference (both days, incl. OpenSQL Camp) is 5 EUR, you can pre-register here until August 10th (and if you do so today, you will still get a free T-Shirt as well!). Of course, you can also just show up at the entrance and pay the entrance fee on site. The OpenSQL Camp will be located in Room C120 - see the instructions on the FrOSCon web page on how to get there and where to find accommodation.

In case you can't make it to Germany for the European Edition, Eric Day and Selena Deckelmann have started to organize another OpenSQL Camp in Portland, Oregon which will take place on November 14th and 15th. More details can be found on the OpenSQL Camp Wiki. Space is limited to 120 attendees, so sign up today and reserve your spot!

We've now published a new set of binary packages including the extended GIS functionality from the mysql-5.1-wl1326 source tree.

This release is based on the MySQL 5.1.35 code base and fixes the bugs mentioned below. It includes some improvements to the GIS functionality as well, so please use these packages for future testing of the MySQL GIS functionality.

The following GIS-related bugs were fixed in this version:

• Bug#31753: Buffer/area functions only return first row of set
• Bug#32032: Contains() does not work on MultiPolygons, may force a disconnect and/or result in extremely long query times.
• Bug#32100: contains, intersects functions never return. Query disconnects or times out.
• Bug#33035: Intersection function returns 'Function doesn't exist' error
• Bug#40874: Buffer function doesn't work with LINESTRING
• Bug#41481: Buffer function never returns, cpu 100%, system locks up
• Bug#43493: union function returns polygon not multipolygon for non-intersecting polygons
• Bug#44753: nan error in union function

I'd like to explicitly thank John Powell for his extensive testing and bug reporting so far, he helped us quite a lot to improve the quality of the GIS code.

Please help us by by testing this new release and reporting bugs at our public bug database. For convenience, please tag your Bug reports with "gis" and make sure to put them in the "Server: GIS" category!